Hewlett Packard Enterprise, the company better known as HPE, announced today that it acquired Pachyderm, a startup developing a data science platform for "explainable, repeatable" AI. The terms of the deal weren't disclosed nor was the purchase price. But HPE said that it plans to integrate Pachyderm's capabilities into a platform that'll deliver a pipeline for automatically preparing, tracking and managing machine learning processes. Pachyderm's software will remain available to current and new customers -- for now, at least. HPE says that the transaction isn't subject to any regulatory approvals and will likely close this month.

Deep learning has enormous promise but developing deep learning models at scale remains extremely complex, time-consuming, and expensive. At Determined AI, part of Hewlett Packard Enterprise, we are working to change that: our revolutionary open-source deep learning training platform enables deep learning engineers to train better models in less time, to seamlessly share GPU clusters, and to collaborate more effectively. They are the main interface between customers and our engineering organization. In this role, you will take on challenging issues that our customers face, working together with other engineers towards a resolution. As a Developer Support Engineer, you can shape our customers' experiences with our products, as well as help define our support processes.

Machine learning PhD students are in a unique position: they often need to run large-scale experiments to conduct state-of-the-art research but they don't have the support of the platform teams that industrial ML engineers can rely on. As former PhD students ourselves, we recount our hands-on experience with these challenges and explain how open-source tools like Determined would have made grad school a lot less painful. When we started graduate school as PhD students at Carnegie Mellon University (CMU), we thought the challenge laid in having novel ideas, testing hypotheses, and presenting research. Instead, the most difficult part was building out the tooling and infrastructure needed to run deep learning experiments. While industry labs like Google Brain and FAIR have teams of engineers to provide this kind of support, independent researchers and graduate students are left to manage on their own.

The News: HOUSTON – June 21, 2021 – Hewlett Packard Enterprise (NYSE: HPE) today announced that it has acquired Determined AI, a San Francisco-based startup that delivers a powerful and robust software stack to train AI models faster, at any scale, using its open source machine learning (ML) platform. HPE will combine Determined AI's unique software solution with its world-leading AI and high performance computing (HPC) offerings to enable ML engineers to easily implement and train machine learning models to provide faster and more accurate insights from their data in almost every industry. Analyst Take: HPE opened up its big Discover 2021 week with a handful of announcements. The deal size wasn't disclosed, but I immediately see this deal as a strategic capability to add to the company's HPC and AI portfolio and something that will be well suited to be incorporated into the company's as-a-service ambitions. Furthermore, the Determined AI acquisition comes at an opportune moment as HPE continues to accelerate the process of transforming its entire portfolio to consumption services as part of its GreenLake portfolio.

A machine learning technology company co-founded by Ameet Talwalkar, an assistant professor in the Machine Learning Department at Carnegie Mellon University's School of Computer Science, will join Hewlett Packard Enterprise (HPE). Determined AI, a San Francisco-based startup, builds software that trains artificial intelligence models more quickly and at scale using its open-source machine learning platform. Talwalkar is chief scientist at Determined AI, which he co-founded in 2017 with Neil Conway and Evan Sparks. "We are thrilled about the opportunity to partner with HPE to deliver co-designed software and hardware and tackle some of society's most pressing challenges," the founders wrote in a blog post announcing the acquisition. "HPE shares our vision that driving an open standard for AI software infrastructure is the fastest way for the industry to realize the potential of AI."

Training complex state-of-the-art natural language processing (NLP) models is now a breeze, thanks to HuggingFace -- making it an essential open-source go-to for data scientists and machine learning engineers to implement Transformers models and configure them as state-of-the-art NLP models with straightforward library calls. As a result, the library has become crucial for training NLP models, like in Baidu or Alibaba, and has contributed to state-of-the-art results in several NLP tasks. Our friends at Determined AI are hosting an exciting lunch-and-learn covering training HuggingFace Transformers at scale using Determined! Learn to train Transformers with distributed training, hyperparameter searches, and cheap spot instances -- all without modifying code. Please consider joining on Wednesday, June 30th at 10 AM PT for a hands-on tutorial from Liam Li, a Senior Machine Learning Engineer at Determined AI, and Angela Jiang, a Product Manager at Determined AI (lunch included!).

More than 20 AI startups have banded together to create the AI Infrastructure Alliance in order to build a software and hardware stack for machine learning and adopt common standards. The alliance brings together companies like Algorithmia; Determined AI, which works with deep learning; data monitoring startup WhyLabs; and Pachyderm, a data science company that raised $16 million last year in a round led by M12, formerly Microsoft Ventures. A spokesperson for the alliance said partner organizations have raised about $200 million in funding from investors. Dan Jeffries, chief tech evangelist at Pachyderm, will serve as director of the alliance. He said the group began to form from conversations that started over a year ago.

Machine learning PhD students are in a unique position: they often need to run large-scale experiments to conduct state-of-the-art research but they don't have the support of the platform teams that industrial ML engineers can rely on. As former PhD students ourselves, we recount our hands-on experience with these challenges and explain how open-source tools like Determined would have made grad school a lot less painful. When we started graduate school as PhD students at Carnegie Mellon University (CMU), we thought the challenge laid in having novel ideas, testing hypotheses, and presenting research. Instead, the most difficult part was building out the tooling and infrastructure needed to run deep learning experiments. While industry labs like Google Brain and FAIR have teams of engineers to provide this kind of support, independent researchers and graduate students are left to manage on their own.

Nearly two years ago, Seattle Sport Sciences, a company that provides data to soccer club executives, coaches, trainers and players to improve training, made a hard turn into AI. It began developing a system that tracks ball physics and player movements from video feeds. To build it, the company needed to label millions of video frames to teach computer algorithms what to look for. It started out by hiring a small team to sit in front of computer screens, identifying players and balls on each frame. But it quickly realized that it needed a software platform in order to scale.

In five lines, you can describe how your architecture looks and then you can also specify what algorithms you want to use for training. There are a lot of other systems challenges associated with actually going end to end, from data to a deployed model. The existing software solutions don't really tackle a big set of these challenges. For example, regardless of the software you're using, it takes days to weeks to train a deep learning model. There's real open challenges of how to best use parallel and distributed computing both to train a particular model and in the context of tuning hyperparameters of different models. We also found out the vast majority of organizations that we've spoken to in the last year or so who are using deep learning for what I'd call mission-critical problems, are actually doing it with on-premise hardware.